Statistical Analysis and Enhancement of Random Testing Methods also under Constrained Resources

Adaptive Random Testing (ART) denotes a family of random testing methods that are designed to be more effective than Random Testing (RT). Mostly, these methods have been investigated using the mean F-measure, which denotes the random number of test cases necessary to detect the first failure. The two most important ART methods, namely Distance-Based ART (DART) and Restricted Random Testing (RRT), perform worse for higher failure rates than for lower failure rates. Furthermore, all previous publications on ART analyzed these methods for testing with unlimited resources. The present paper investigates, why DART and RRT behave better for lower failure rates. Therefore, the F-measure distribution and the spatial distribution of single test cases are analyzed. Thereby, shortcomings of D-ART and RRT are revealed. Improved ART methods are presented based on our findings. Furthermore, the usefulness of the F-measure distribution of testing with unlimited resources for resourceconstrained testing is explained. Finally, the ART methods are compared to RT for both cases, i. e. with and without resource limitations.